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Abstract 

We explore a simplified class of models we call swarms, which are 
inspired by the collective behavior of social insects. We perform a 
mean-field stability analysis and perform numerical simulations of the 
model. Several interesting types of behavior emerge in the vicinity 
of a second-order phase transition in the model, including the for- 
mation of stable lines of traffic flow, and memory reconstitution and 
bootstrapping. In addition to providing an understanding of certain 
classes of biological behavior, these models bear a generic resemblance 
to a number of pattern formation processes in the physical sciences. 



1 The Model 

The self-organization of patterns of flow in social insect swarms is a beau- 
tiful example of how intelligent and efficient behavior of the whole can be 
achieved even in the absence of any particular intelligence or forethought of 
the individuals. Q Indeed, such patterns can have functionality even without 
the awareness of the individual entities themselves. A study of the essential 
elements of swarm dynamics provides an understanding of such behaviors, 
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and is our principal goal in this letter. A secondary, broader goal is to study 
the generic behavior of a kind of stochastic particle-field system in which the 
motion of the "particles" both changes the field and is affected by the field. 
There are a number of systems like this in the physical sciences. We partic- 
ularly have in mind systems where the relaxation time scales of the field are 
slow in comparison to the motion of the particles and this is what makes the 
present work unique and interesting. 

Consider the following scenario. Suppose particles move in a field cr(x) 
that field corresponds to an energy £7[cr(x)] = U(x) so that each particle 
experiences a force due to the field of F = — VZ7(x). Suppose also that the 
particles are subject to random perturbations so that their motion is only 
statistically in the direction of F. We could describe their motion by the 
stochastic (Langevin) type equation 



where f(t) is Gaussian white noise with < f(t) >= and < f(£) • f(s) >= 
(l//3)15(t — s). The parameter T is a friction coefficient. Obviously for social 
insects this term is not to be taken literally. The "friction" in such cases 
merely parameterizes the tendency of the organism-particles to continue in a 
given direction. A smaller "friction" thus means that the organism's velocity 
vector is correlated over longer times. In physical systems this parameter 
may have a more literal meaning. 

The noise strength is 1//3. Thus (3, which we call the osmotropotaxic 
sensitivity^, determines the degree of determinacy with which the organism- 
particle follow the gradient of the field F(x) = — V£/(x) = — £/'[<r(x)] Vcr(x). 
If (5 is large the predominant force felt by the particles is F, and they fol- 
low F more accurately. If (3 is small the noise becomes more important 
and the organism-particles follow the field gradient with less certainty. It is 
necessary to assume probabilistic laws in order to reproduce the behaviors 
observed experimentally, and the osmotropotaxic sensitivity can be, and has 
been, observed experimentally. []5], ||, As we will see, the degree of random- 
ness plays a very important role in the functionality of the swarm. Most 
probabably it is hardwired into the sensory apparatus of the organism. 

Up to this point we have simply described the noisy motion of particles in 
some "energy landscape" . The principal element of interest here is that the 



x + Ix + W[a(x)] = f(t). 



(1) 
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energy landscape will be allowed to evolve in response to the noisy motion of 
the particles. We will choose one of the simplest possible models for how this 
occurs. H |Tt], [TTJ The presence of a particle at x will "deposit" an amount 
of field 77 per unit time. For insect swarms cr(x) is to be interpreted as the 
pheromonal field density |L2| at x. An amount of field kct(x) will "decay" per 



unit time. The field might also diffuse in space with diffusion constant D 
such that the field will evolve according to 

^= J DVV(x)+r /P (x)-^(x) (2) 

where p(x) is the particle density at x. When the particles are social insects 
this equation describes the evolution of pheromonal field a laid down by the 
organisms as they walk, and the parameter k is an evaporation rate. 

The only thing which has been left unspecified here is U[a\. In this letter 
we use 

W = ->»( 1 + TT^)- (3) 

The use of this "field energy function" has already been justified elsewhere, |9|, 
|T0| , |TT| where it has been explained at length how this function is derived from 
the experimentally observed behavior of real ants. Other energy functions can 
be used for different physical situations, and the general framework of this 
model is capable of supporting many different types of systems. 

Here we call 1/5 the capacity. This models the fact that an actual ant's 
response to additional concentration of pheromone decreases somewhat at 
high concentrations. This gives rise to a peaked function for the average time 
an ant will stay on a line of ant traffic as the concentration of pheromone is 



varied - a fact which has been repeatedly observed experimentally [jT3| , [L4| [15 
This is perhaps significant in light of one of the results presented here: trails 
and networks do not spontaneously form in the absence of this saturation 
effect. One reasonable physiological explanation for this effect is that the 
ant's sensing organs become saturated: since each antenna has only a finite 
number of pheromone receptor sites, the antennae are effectively jammed at 
high concentration, and the response to gradients can be expected to grow 
less pronounced. 

The model described above was constructed precisely because it approxi- 
mately reproduces the "microscopic" behavior of individual ants as observed 
in the laboratory.^], |3], f|, 0] Notice that we have resisted the temptation to 
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write a Fokker-Planck equation for p(p, x, t) from Eq. 1. This is to empha- 
size that we do not want to consider the continous limit, but rather the case 
where internal fluctuations are significant. Since the field a corresponds to 
the pheromone which, unlike the ant density p, is composed of a macroscop- 
ically large number of particles, we can use the continuous Equation 2 to 
describe it. We have expressed the model in this particularly suggestive form 
to emphasize its physical aspects, and its generic relationship to other phys- 
ical systems. We will not expand on this point here, but will merely indicate 
the generic resemblence of this model to problems such as anomalous ionic 
diffusion in polymeric materials, stochastic growth processes, the evolution 
of river basins, and other types of complex physical systems that incorporate 
mobile elements and an evolving substrate of some kind. |T6j 



2 Analysis 

Since a detailed mathematical analysis of this type of system has already 
been made|| [TTJ, we will confine ourselves here to a brief statement of the 
pertinent results. 

We analyze the case where the relaxation of the field er(x) is very slow 
in comparison to the relaxation time of the particle density p(x). In this 
case the field is said to slave [|18j to the particle denisty, and the particle 
density can be removed from the picture in the following way. Since the field 
changes very little on these time scales we will assume it is constant as the 
particles equilibrate in the energy landscape £/[cr(x)]. It is easy to show that 
the equilibrium distribution of particles evolves to 

pe[aWl J d°ye*p{-flU[*(y)]) W 

Elimination of the particle variables is made possible by substituting this 
expression into Eq. 2, so that 

da(x,t) _ V Nexp(-f3U[a(x,t)}) 

dt 'J d°ye*p(-{lU[*(y,t)}) K0Wj ' W 

where N is the total number of particles, and where we neglect diffusion and 
will not consider it futher in the present letter. For our present purpose we 
will only be interested in the stability of the uniform phase of the system, 
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<t(x, t) = cr(t), which we will treat by mean- field theory. The above equation 
has a mean-field solution of <jq = Ntj/Vk, where V = J d D y. 

The stability of this solution is found by expanding about the solution 
with er(x, t) = a + <5cr(x, t). This leads to the mean-field stability criterion 

f3a U'[a } + 1 > 0. (6) 

This is the most important theoretical result, and it gives the location of the 
second-order phase transition, that is, the location of the boundary separat- 
ing totally random behavior from ordered behavior of varying types in the 
physiological phase space. As shown in || [H| , ordered behavior sets in when 



this criterion is broken. This criterion is true for any energy function U[a], 
and allows us to calculate the physiological phase boundaries in every case. 

For the particular behavioral energy function used here (Eq. 3) the tran- 
sition lies along the curve 

p c (5, (T ) = 1 + l/ao + 25 + 5a + 5 2 a . (7) 

The symmetry which allows for a mean-field type solution for the location 
of the transition line case (detailed balance) is maintained up to the point 
of the transition and is effectively spontaneously broken at that point. This 
means the points of transition from disorder to order can be determined 
theoretically, but not the resulting patterns, which must be determined via 
simulations or further theoretical analysis. 

In addition to being the major landmark in the physiological parame- 
ter space, we believe both on general grounds, and because of the results 
presented in the next sections, that the location of this line has significant 
behavioral implications. Clearly it is important that ordered behavior exists 
so that ordered patterns of flow can form, but just as importantly, the be- 
havior should not be too rigid and ordered since fluctuation and instabilities 
might increase the flexibility of response of the mass action. Thus, if the 
dynamics are such that there can be significant fluctuations in the patterns 
of mass action, the swarms could better respond to a changing environment. 
We might conclude that a "good" place to be would be in the order region, 
but near to the transition line in such a way as to optimize the conflict- 
ing tendencies of controlled order behaviors versus flexible random behavior. 
We will also see in the next sections that large fluctuation actually serves to 
stabilize some of the important patterns of collective behavior of the system. 
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3 Numerics 



Equations 1, 2 and describe the system, but for the purposes of simulation 
it is necessary to introduce some discretization of space and time, and to 
translate the noisy behavioral function observed experimentally into transi- 
tion rules on this discrete space |§. These discrete rules are merely tools 
for the approximation of a continuous model, and other discretizations are 
possible. 

The agent (referred to as an "ant") will be allowed to move from site to 
site on a square lattice. We allow each ant to take one step on the lattice 
of points (cells) at each time step. As a result of discretizing the space, 
an individual ant at each time step finds itself in one of these cells, and its 
sensory input is the concentration of pheromone in its own cell and each of 
the eight neighboring cells. In addition, each ant leaves a constant amount 
of pheromone at the node in which it is located at every time step. This 
pheromone decays (is reduced by a certain percentage) at each time step. 
Toroidial boundary conditions are imposed on the lattice to remove, as much 
as possible, any boundary effects. The transition rates from cell % to cell j are 
proportional to W oc exp(—/3(U[<jj]). The normalized transition probabilities 
on the lattice to go from cell k to cell % are then given by 

where the notation Tlj/k indicates the sum over all the cells j which are in 
the local neighborhood of k. Aj measures the magnitude of the difference in 
orientation (direction) to the previous direction the last time the ant moved. 
Since we are using a neighborhood composed of the cell and its eight neigh- 
bors on a square lattice, A, can take only the discrete values — 4, and 
it is sufficient to assign a number Wi for each of these changes of direction. 
Here we used weights of (same direction) W = 1, and W\ = 1/2, w 2 = 1/8, 
w 3 = 1/12 and w 4 = 1/50 (u-turn). Once the parameters (3, 5 and u>j are set 
a large number of ants can be placed on the lattice at random positions, the 
movement of each ant can be determined randomly taken from the distribu- 
tion given by Pik- We usually take the initial condition of the pheromone 
to be zero at every point on the lattice. Every time step each ant leaves a 
quantity rj (here rj = 0.05) of pheromone in each cell, and the total amount 
of pheromone <7j in each cell is decreased at a rate k (here k = 0.015) at the 
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end of each time step. The evolution of the system is simulated numerically 
and the pattern of lattice sites that contain ants is displayed in the figures. 

4 The Physiological Phase-Plot 

We studied a wide range of scent laying and decay rates, as well as differ- 
ent densities of ants. This is the simplest local, memoryless, homogeneous 
and isotropic model which leads to trail forming that we know of, and the 
formation of trails and networks of ant traffic is not imposed by any special 
boundary conditions, lattice topology, or additional behavioral rules. The 
required behavioral elements are stochastic, nonlinear response of an ant to 
the scent, and a directional bias. Furthermore the parameters of the system 
need to be tuned somewhat to the appropriate region, and not any nonlinear 
rule of the above type will do. If the nonlinear response or the directional 
bias are removed no lines form, and lines that are already formed do not 
persist. 

Well defined trails form in the region above the phase transition line 
(ordered phase), but near the transition from disorder. Further away from 
the order- disorder line the clumping tendency overcomes the directional bias, 
and no lines form. 

In every run observed within the part of the parameter space indicated in 
Figure 4, a final system of trails emerged and survived as long as the simula- 
tion was run (Fig. 1). The self-organization of regular patterns of movement 
of the ants occurs in two stages: an initial condensation phase, which occurs 
during approximately the first 500 time steps, and a simplification phase. In 
the first stage a network of trails forms within a few hundred time steps. The 
trails at first typically form a network of many branches. All of the branches 
initially appear stable, but after several hundred time steps, some abruptly 
disappear, decreasing the length of the network. 

In this second stage the network of ant traffic resembles qualitatively some 
network type experiments done with real ants|7[] and the network theory of 
|J, (and the basic discussion of the properties of these networks given there 
holds). The ants can neglect a branch by chance, and if the fluctuation in the 
density of the ants over the branch happens to be great enough, the neglect 
builds on itself and the trail disappears. As a result, the complexity of the 
network tends to decrease with time until a final, stable network is reached. 
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The final line in all runs has been a loop (sometimes with sub- loops). 
Most often, the trail exploits the periodic boundary conditions by forming a 
more-or-less straight line that wraps around the torus. This of course reflects 
the situation that a line "free end" is clearly unstable due to the directional 
bias. That lines which feed back on themselves invariably form is thus not 
surprising. It is interesting to note here the qualitative similarity of these 



loops to autocatalytic sets of chemical reactions. JT9|, |2(J The instability of 
the free ends just illustrates the sensitivity to boundary conditions when the 
information flows in an active way. The separate issues of how these basic 
structures are incorporated into a larger colony context will be taken up 
elsewhere, and we believe they deserve further mathematical analysis. 

The main conclusion to be drawn here is that simple osmotropotaxic 
scent following of the very simple kind described above is no only sufficient 
to allow for trail following behavior as shown in |3], Q , but sufficient to produce 
evolution of complex pattern of organized flow of social insect traffic all by 
itself 



5 Consolidation, Memory, and Bootstrapping 

Since the self-organizing properties of the swarm are instability driven, the 
structures that form have some very interesting properties with respect to 
large perturbations. We performed the following experiment: the system was 
tuned to a region of the phase plane where lines form, and a stable network of 
traffic in a straight line was set up. Then the noise level was increased so that 
system is tuned below the transition line ((3 < (3 C ). One observes that the 
ants fall away from their orderly patterns and immediately start executing 
random walks on the lattice. As a result the pheromone distribution starts 
to fluctuate and become more and more random. If f3 is then tuned back to 
its original value at some time later the line will eventually reform with little 
or no change. This occurs even if the randomization is allowed to proceed to 
the point that the pheromonal field is almost totally randomized. 

Fig. 3 is shows a measure of how much of the original pattern in the 
pheromonal field is left over time as the ants execute a random walk for a 
certain time and then go back to nonlinear scent response. We define 7r as 
the number of lattice sites where the concentration of pheromone is above 
the average for the grid in both the original pattern and the current field, as 
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a percentage of the number of current sites with above average pheromone. 
Thus 7r is a measure of the overlap of the current pattern with the original 
pattern. The figure plots n for different durations of the random walk, and 
shows the stable patterns that re-form after the nonlinear scent response is 
turned back on. For times up to about twice the decay time r = 1/k even 
small, virtually undetectable memory effects of the field can be amplified 
causing the patterns to reform without significant changes. As shown in Fig. 
3, as much as 90% of the pattern can be erased, and entirely reconstituted 
later. 

To further show the adaptability of the swarm in certain regions of the 
parameter space, the following experiment was performed. The system was 
tuned to a region of (3-5 parameter space where trails never form, and the 
motion of the ants remains random. A weak "bootstrapping" trail was then 
added to the grid, the pheromone density of which was on the order of some 
of the larger random fluctuations in the field. In a region slightly below the 
phase transition, this causes the swarm to organize and amplify the trail, 
in spite of the fact that a network would not have formed spontaneously. 
Fig. 4 plots a measure of pheromone concentration v, which is the ratio 
of lattice sites with below-average pheromone concentration to those with 
above-average concentration. Three runs are shown; one where no boot- 
strapping trail is added; another in which a bootstrapping trail causes the 
swarm to shift into the trail, and lastly, an attempt at bootstrapping which 
failed because the swarm was too far from the phase transition line. We 
emphasize that this behavior appears below, but near enough to the phase 
transition line. 

Memory reconsolidation and bootstrapping are two seemingly conflict- 
ing functional abilities which appear in the vicinity to the phase transition 
line. Firstly, these organized patterns of behavior are really quite stable with 
respect to large perturbation, s which might have an obvious usefulness to 
operation in a changing and unpredictable environment. Secondly, because 
the patterns are due to the formation of an initially weak stable coopera- 
tive structure, the swarm can act as an information amplifier, and even a 
weak external perturbation (such as the location of a food source by a single 
ant) might lead to a significant response. Thus swarms posses both a long 
memory and the ability to learn. 
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6 Conclusions 



As shown in Fig. 2, lines of traffic form near the second-order phase transition 
line. In this region there is a cooperative effect between the fluctuations and 
the ordering effects. Much below the transition line the fluctuations are too 
great, and no cooperative structures form. However, too far above the tran- 
sition line the fluctuations are suppressed, and order dominates overwhelm- 
ingly. This means that the ants are basically induced to "turn around" with 
sufficient probability that only patches form. The role of such "U-turns" , has 



been investigated experimentally and theoretically in 21 , where it was also 



suggested that they might play an important role in the self-organization of 
ant traffic. 

The region in which the lines form represents a cooperative effect between 
the fluctuations and the ordering behavior. The formation of lines can also 



be understood in terms of the theory in |9], pTOfl . In the region of the lattice 
where there are lines, the clumping effect is both unstable in the transverse 
direction and stable in the lateral direction of the particle motion, and it is 
this fact which gives rise to the formation of stable lines of traffic. The result 
is a fairly robust region of line formation. Motion in this region can perhaps 
be likened to motion in the liquid crystal phase of matter, or the reptation of 
polymer chains in polymer melts, in which the order along one axis is frozen 
in, while along another motion can occur freely. 

Since information (in the form of the orderly movement of ants from one 
place to another) can be said to flow only in the region of stable line forma- 
tion, which lies above the disordered region and below the very ordered re- 
gion, these results might mistakenly be thought similar to the "meta-theory" 
of "complexity at the edge of chaos" which asserts that complex behavior 



emerges in the vicinity of a marginally stable state. [22, E3l E4L B5| However 



we point out that the system, as far as we know, does not "self-organize" to 
this region unless it may be said that on the evolutionary time scale the bi- 
ological organisms found such behavior adaptive. In addition there is not an 
"edge" but rather a large robust region where the most complex structures 
form. Lastly we do not believe that the reason for this type of behavior has 
anything to do with the very speculative ones which are sometimes suggested. 
It is quite natural that complex behaviors resulting from the competition be- 
tween stable and unstable modes should appear near a phase transition line, 
since the first unstable modes appear there. Such behaviors can be incorpo- 
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rated as biological functionality as shown in the few examples discussed in 
this letter, but this functionality has nothing to do with the notion of com- 
plexity "at the edge of chaos" and the hypothetical universal computational 
properties which may or may not exist there. [p3|, p5| 

ER would like to thank the the Santa Fe Institute and the NSF REU 
program, and the UGS program at LANL which supported parts of this 
research. 
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Figure 1: Trail formation. A: shows the swarm at time 29, when the field is still 
mostly random. B: at t — 208 C: at t — 332, the network has initially formed, 
D: at t — 412. E: at t — 785, where some branches have disappeared. F: at 
t = 1753, shows the final network that emerges. 



Figure 2: Physiological phase plot in space for cr = 1.367. Figure show 
the distribution of particles after 5000 time steps centered on the values of the 
physiological parameter for that run. 



Figure 3: Memory erasure and reconsolidation. A: The original pattern. B: (3 is 
turned back up after 250 time steps; the trail re-forms completely. C: (3 is turned 
back up after 265 steps; only a portion of the trail remains. D: f3 is turned back 
up after 275 steps. E: j3 is turned back up after 300 steps. F: (3 is turned back 
up after 375 steps. 



Figure 4: The "bootstrapping" effect. The Plot show the onset the effect of 
the introduction of a bootsrapping trail. Plot A shows the random configuration 
of the ants before the trail is added. B: shows the swarm in the short period of 
amplifying this weak trail. C: the final state. 
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